Image processing apparatus and method thereof

ABSTRACT

Chroma correction is executed in consideration of the area effect of colors. To this end, an input image is segmented into a plurality of regions as sets of pixels having similar feature amounts, and a size evaluation value of the each segmented regions is calculated based on the number of pixels and a shape of the each segmented region. Chroma correction is applied to the input image to weaken correction as the region has a larger size evaluation value.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to image processing for chroma-correctingan image.

2. Description of the Related Art

Chroma correction processing that corrects the chroma of an image toenhance the brightness of the image and to improve appearance of theimage is available.

On the other hand, as a human visual characteristic, a phenomenon calledan area effect of colors is known. The area effect of colors is aphenomenon in which one may perceive different lightness and chromalevels of objects depending on their apparent sizes. For example, evenwhen objects have the same color, when one object has a large apparentsize in an image, one perceives it to have higher lightness and chromalevels than the other objects with a smaller apparent size. Due to thisarea effect of colors, a phenomenon in which one perceives differentobject colors in an image depending on their different visual sizesoccurs. For example, if there are regions having the same color butdifferent sizes, one perceives that the larger region has a higherchroma level and is brighter than the smaller region.

As the chroma correction processing, a technique for applying chromacorrection to respective pixels of an image according to an amount ofchroma correction is available. With this technique, upon uniformlyapplying chroma correction to an image, a desired effect can be obtainedon image data. However, the chroma level changes depending on theapparent sizes of regions due to the area effect of colors, and adesired effect cannot often be obtained. That is, chroma correction mustbe applied according to the apparent sizes of regions.

Japanese Patent No. 3370770 discloses an invention that corrects achroma level according to the area of a human skin color in an image.More specifically, that invention detects the area of a skin colorregion (to be referred to as a skin color area hereinafter) in one framefrom color-difference signals, detects the average luminance of theframe from a luminance signal, and determines the amount of skin colorcorrection of that frame based on the average luminance and skin colorarea. Then, the invention corrects a luminance signal andcolor-difference signals corresponding to the skin color regionaccording to the amount of skin color correction.

Japanese Patent Laid-Open No. 2004-192614 discloses an invention thatcorrects a memory color included in image data into a favorable color.More specifically, this invention calculates characteristic values fromimage data, and extracts a specific color region of a correction targetand its background region from the characteristic values and colorinformation of the memory color. The invention calculates a target valueof color correction of a favorable color based on the characteristicvalues, extraction result, and favorable color information, and correctsrespective colors of the specific color region based on the target valueof color correction and characteristic values. The characteristic valuesinclude the average chromaticity of the specific color region, and theoccupation ratio of the specific color region to an image area.

However, if there are a large number of small skin color regions in oneframe, the technique of Japanese Patent No. 3370770 determines an amountof skin color correction according to the total of areas of these skincolor regions, and applies uniform correction to the respective skincolor regions. Therefore, this technique cannot appropriately correcteach individual skin color region. Also, the technique of JapanesePatent No. 3370770 does not apply color correction to regions other thanskin color regions, and the technique of Japanese Patent Laid-Open No.2004-192614 does not apply color correction to color regions other thanthe memory color.

SUMMARY OF THE INVENTION

In one aspect, an image processing method comprising the steps ofsegmenting an input image into a plurality of regions as sets of pixelshaving similar feature amounts; calculating a size evaluation value ofeach segmented region based on the number of pixels and a shape of theeach segmented region, and applying chroma correction to the input imageto weaken correction as a region has a larger size evaluation value.

According to the aspect, chroma correction can be implemented inconsideration of the area effect of colors.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the arrangement of an image processingapparatus according to one embodiment;

FIG. 2 is a flowchart for explaining the processing to be executed by acontroller;

FIGS. 3A and 3B are flowcharts for explaining the processing to beexecuted by a region segmentation unit;

FIG. 4 is a flowchart showing the processing of a size calculator;

FIG. 5 is a flowchart showing the processing of a correction coefficientcalculator;

FIG. 6 is a graph showing an example of a function used to calculate acoefficient of chroma correction;

FIG. 7 is a flowchart showing the processing of a chroma corrector;

FIG. 8 is a block diagram for explaining the relationship betweenrespective processors and storage areas;

FIG. 9 shows image data of an input image;

FIG. 10 shows a region segmentation result of the image data shown inFIG. 9;

FIG. 11 shows region information of respective regions;

FIG. 12 shows coefficients of chroma correction of respective pixels;

FIG. 13 shows image data after chroma correction;

FIG. 14 shows a region segmentation result of the image data shown inFIG. 9;

FIG. 15 shows region information of respective regions;

FIG. 16 shows coefficients of chroma correction of respective pixels;

FIG. 17 shows image data after chroma correction;

FIG. 18 shows a data example stored in a correction coefficient storagearea after filtering;

FIG. 19 is a flowchart showing the processing of a size calculatoraccording to the second embodiment;

FIGS. 20 to 23 are flowcharts showing detailed processing of step S2210shown in FIG. 19; and

FIG. 24 is a flowchart showing the processing of a region segmentationunit according to the third embodiment.

DESCRIPTION OF THE EMBODIMENTS

An image processing apparatus according to embodiments of the presentinvention will be described in detail hereinafter with reference to theaccompanying drawings.

First Embodiment

[Arrangement of Image Processing Apparatus]

FIG. 1 is a block diagram showing the arrangement of an image processingapparatus according to one embodiment.

A region segmentation unit 210 segments an input image into regions. Asize calculator 220 calculates the apparent sizes of respective regionsas size evaluation values of the regions. A correction coefficientcalculator 230 calculates coefficients of chroma correction according tothe apparent sizes of respective regions. A filter processor 240executes filtering according to the coefficients of correction. A chromacorrector 250 outputs an image that has undergone chroma correction. Aninput unit 270 is an interface which receives an image from an externaldevice, and an output unit 260 is an interface which outputs an image toan external device.

A controller 200 dynamically assigns an area for storing an input imageand a work area to a memory 201, controls the aforementioned processorsto execute image processing to be described later via a system bus 202.Note that the memory 201 comprises a semiconductor memory. Also, amemory such as a hard disk or the like may be used as an auxiliarystorage device.

As will be described in detail later, the size calculator 220 calculatesa value indicating the apparent size of each region. Since the apparentsize changes depending on the shape of the region even when the numberof pixels remains the same, information indicating the apparent size isrequired. For example, a flat region looks small even when it has alarge number of pixels.

In the following description, as for the pixel address of an image, xrepresents the horizontal direction, y represents the verticaldirection, and the upper left pixel address of an image is (x, y)=(0,0). If an image size is defined by M×N pixels, the address of the lowerright pixel of the image is (x, y)=(M, N).

[Controller]

FIG. 2 is a flowchart for explaining the processing to be executed bythe controller 200.

The controller 200 assigns the following areas to the memory 201(S2001). That is, the areas to be assigned include an image storage areafor storing an input image, a label storage area for storing labelnumbers appended to image regions, a region information storage area forstoring the information of the areas and positions of respective imageregions, a correction coefficient storage area for storing coefficientsof correction, a work area used for respective processes, and the like.

The controller 200 initializes the storage areas assigned to the memory201 (S2002). In this case, the controller 200 writes an initial value(e.g., 0) of corresponding information. However, an arbitrary value maybe used as the initial value as long as it does not influence subsequentprocesses.

The controller 200 stores an image input from the input unit 270 in theimage storage area (S2003). The image may have any of an LCh (lightness,chroma, hue) data format, RGB data format, andluminance/color-difference signal format such as YCbCr or the like. Inthe following description, assume that the image data is YCbCr data.

Upon storing an image for at least one frame in the image storage area,the controller 200 controls the respective processors to execute imageprocessing (S2004).

[Region Segmentation Unit]

Region segmentation is processing for segmenting one image into aplurality of image regions as sets of pixels having similar featureamounts. The feature amounts include color information such aslightness, chroma, hue, and the like, image signal information (RGBprimary color signals, or YCbCr luminance and color difference signals),texture information, and the like. Upon processing an image oftelevision broadcasting, YCbCr signals of a video signal are preferablyused to obviate the need for conversion and the like.

As the region segmentation method, for example, a region growing method,statistical hypothesis testing method, integration method, separationmethod, separation/integration method, K-means method, and the like maybe used. In this embodiment, since counting of the number of pixels ofeach region and detection of the top, bottom, right, and left ends (tobe referred to as “ends of every directions” hereinafter) are donesimultaneously, the region growing method is suitably used.

In the region growing method, a given pixel is defined as an origin, andthe feature amount of that origin pixel is used as that of a region. Ifthe difference between the feature amount of a pixel adjacent to theorigin pixel and that of the region falls within a predetermined range,it is determined that the pixel adjacent to the origin pixel belongs tothe same region as the origin pixel, and the same label as the originpixel is appended to the pixel adjacent to the origin pixel.Subsequently, as for each pixel adjacent to the region (to be referredto as an “adjacent pixel” hereinafter), feature amount comparison ismade to grow the region. If no adjacent pixel with a close featureamount remains, segmentation of one region is completed.

A method of selecting an origin pixel is arbitrary. For example, it isconvenient to raster-scan pixels, and to select the first pixel to whichno label is assigned as an origin. Respective regions segmented in thisway are assigned labels with different numbers. A label with the samenumber is appended to an adjacent pixel determined to belong to the sameregion. The label numbers are stored in the label storage area forrespective pixels.

FIGS. 3A and 3B are flowcharts for explaining the processing to beexecuted by the region segmentation unit 210.

The region segmentation unit 210 sets a label number L to be 1 (S2101),and also sets a count value N to be 1 (S2102). The unit 210 thendetermines an origin pixel and acquires a feature amount of the originpixel (S2103). The unit 210 sets the feature amount of the origin pixelas that of a focused region (S2104). The unit 210 then initializes data(to be referred to as “region range data” hereinafter) indicating thepositions of the ends in every directions of the focused region based onthe position of the origin pixel (S2105). The region range data isdefined by an address (X0, Y0) of the upper left end of the regions andthat (X1, Y1) of the lower right end. Initially, an address (x, y) ofthe origin pixel is set in both the addresses.

The region segmentation unit 210 acquires the feature amount of anadjacent pixel (S2106), and checks if the difference between the featureamounts of the adjacent pixel and the focused region falls within apredetermined range (S2107). As a result of checking, if the differencefalls outside the predetermined range, the process jumps to step S2112.

If the difference between the feature amounts of the adjacent pixel andthe focused region falls within the predetermined range, the regionsegmentation unit 210 increments the count value N (S2108). The unit 210then checks if the adjacent pixel is located outside the focused region(a range indicated by the region range data) (S2109). If the adjacentpixel is located outside the focused region, the unit 210 updates theregion range data based on the position of the adjacent pixel (S2110).Let (x1, y1) be the address of the adjacent pixel which is determined tobelong to the focused region. Then, if X0≦x1≦X1 and Y0≦≦Y1, since theadjacent pixel is located within the focused region, the region rangedata need not be updated. However, if x1<X0 and Y0 ≦y1≦Y1, since theadjacent pixel is located on the left side of the left end of thefocused region, X0 of the address (X0, Y0) of the upper left end of theregion is updated to x1. Subsequently, the region segmentation unit 210stores the label number L in a label storage area corresponding to theposition of the adjacent pixel (S2111) to label the adjacent pixel andto add it to the focused region.

The region segmentation unit 210 checks if a pixel which is an adjacentpixel of the focused region and whose feature amount is not acquired yetstill remains (S2112). As a result of checking, if a pixel whose featureamount is not acquired yet still remains, the process returns to stepS2106 to repeat the processes in steps S2106 to S2111.

If no adjacent pixel whose feature amount is not acquired yet remains,the region segmentation unit 210 compares the count value N with athreshold Nth (S2113). As a result of comparison, if N≧Nth, the unit 210stores the count value N in the region information storage area as thenumber NL of pixels and also the region range data in the regioninformation storage area (S2114), and increments the label number L(S2115). If N<Nth, the unit 210 rewrites an element that stores thelabel number L in the label storage area to zero (S2116). Note that thelabel number “0” represents a pixel which does not belong to any region.Also, if N<Nth, the unit 210 does not update the label number L and usesit for labeling of the next region.

Since the threshold Nth is set to be a value obtained by dividing thetotal number of pixels of the image by a desired number of segmentedregions, the number of segmented regions preferably falls within adesirable range. For example, upon segmenting an image of 1920(horizontal)×1080 (vertical) pixels into a maximum of 250 regions, since1920×1080÷250=8294.4, assuming Nth=8295 by counting fractions as one,the image is segmented into a maximum of 250 regions.

Next, the region segmentation unit 210 checks if the region segmentationis complete (S2117). When determining the origin pixel by the rasterscan, completion of the region segmentation can be determined bychecking whether or not the raster scan reaches a pixel at the lowerright end of the image. Alternatively, completion of the regionsegmentation may be determined by checking if label numbers are assignedto all pixels. In this case, however, the initial value of the labelstorage area must be set to be a value other than zero and an expectedlabel number. If it is determined that the region segmentation is notcomplete yet, the process returns to step S2102, and the unit 210executes the next region segmentation. Note that an origin pixel of thenext focused region is a pixel which is adjacent to the right side ofthe immediately preceding focused region. However, when the origin pixelposition has reached the right end of the image, the next origin pixelis a pixel at the left end of the image.

If it is determined that the region segmentation is complete, the regionsegmentation unit 210 stores a maximum value Lmax of the label number Lin the region information storage area (S2118), thus ending theprocessing. When the initial value of the region information storagearea is set to be zero, the number N₀ of pixels with the label numberL=0 is zero. However, when the initial value is set to be a value otherthan zero, the number N₀ of pixels must be set to be zero at that time.The size calculator 220 similarly processes the region with the labelnumber L=0 in the same manner as other regions. In this case, by settingthe number N₀ of pixels=0 of the region with the label number L=0, thecalculation result about the set of pixels which do not belong to anyregion can be zero.

In the description of the above example, the number of pixels and rangeof each region are updated while executing the region segmentation.Alternatively, after segmentation of one region, or after completion ofthe region segmentation, the number of pixels and range of each regionmay be acquired. In the description of the above example, aftersegmentation of one region, the number of pixels of that region ischecked, and if it is smaller than the threshold, the processing forrewriting the label number is executed. Alternatively, after completionof the region segmentation, the rewrite processing may be executed.

[Size Calculator]

FIG. 4 is a flowchart showing the processing of the size calculator 220.

The size calculator 220 acquires the maximum value Lmax of the labelnumber from the region information storage area (S2201), and initializesa counter M to zero (S2202).

Next, the size calculator 220 acquires the number N_(M) of pixels andregion range data of a region corresponding to the value of the counterM from the region information storage area (S2203), and calculates theapparent size of that region (S2204). A calculation example of theapparent size will be described below.

In the following description, various kinds of information of a givenregion A are expressed by the following symbols and the calculations ofthe apparent size will be explained.

V: apparent size (size evaluation value)

S: number N_(A) of pixels

T: top end position Y0 _(A)

B: bottom end position Y1 _(A)

L: left end position X0 _(A)

R: right end position X1 _(A)

Also, let Max( ) be a function that outputs the maximum one of numericalvalues in parentheses, and Min( ) be a function that outputs the minimumone of numerical values in parentheses.

The size calculator 220 calculates the width W and height H of theregion first.W=R−L+1H=B−T+1  (1)

The size calculator 220 then calculates the apparent size V based on Wand H. In this case, the following calculation methods are available.V=S×Min(W,H)/Max(W,H)  (2)V=S×(W+H)/{2×Max(W,H)}  (3)V=S×√S/Max(W,H)  (4)V=S×{Min(W,H)/Max(W,H)}×{S/(W×H)}  (5)

According to one of equations (2) to (5), the apparent size V can besimply calculated. However, since a calculation result roughlyproportional to the apparent size need only be obtained, informationother than the number of pixels and region range data may be used. Aswill be described in detail later, using features of the region shapeobtained from the information stored in the label storage area, e.g.,the width Wc and height Hc of a region central portion, the apparentsize V may be calculated by:V=S×Min(Wc,Hc)/Max(Wc,Hc)  (6)

Since the apparent size V of the region is calculated (S2204), theinfluence of appearance due to the region shape can be preventedcompared to a case wherein the size of the region is simply determinedbased only on the number of pixels, thus obtaining information roughlyproportional to the size of the region. According to equation (6), theapparent size V grows as large as the number of pixels S of the regionincreases, and that grows as large as the shape of the region approachesa round shape.

Next, the size calculator 220 stores the apparent size V in the regioninformation storage area (S2205), increments the counter M (S2206), andcompares M and Lmax (S2207). If M≦Lmax, the process returns to stepS2203; if M>Lmax, the processing ends.

Note that the apparent sizes of respective regions may be calculated atthe same time after completion of the region segmentation by the regionsegmentation unit 210. Alternatively, the apparent size may becalculated every time one region is segmented.

[Correction Coefficient Calculator]

FIG. 5 is a flowchart showing the processing of the correctioncoefficient calculator 230.

The correction coefficient calculator 230 calculates a coefficient ofchroma correction indicating the degree of chroma emphasis to be appliedto an image. If the coefficient of chroma correction is large, strongchroma emphasis is applied to an image; otherwise, weak chroma emphasisis applied to an image. Of course, if the coefficient of chromacorrection is zero, no chroma emphasis is applied to an image.Therefore, in order to correct the area effect of colors with which thechroma of a region with a larger apparent size looks high compared tothat of a region with a small apparent size, the coefficient of chromacorrection is set to be smaller for a region with a larger apparentsize.

The correction coefficient calculator 230 acquires the maximum valueLmax of the label number from the region information storage area(S2301), and initializes a counter M to zero (S2302).

The correction coefficient calculator 230 acquires the apparent size Vof a region corresponding to the value of the counter M from the regioninformation storage area (S2303), and calculates a coefficient C ofchroma correction (S2304).

FIG. 6 shows an example of a function used to calculate the coefficientof chroma correction. In FIG. 6, the abscissa plots the apparent size V,and the ordinate plots the coefficient C of chroma correction.

As shown in FIG. 6, if V≦Th1, the coefficient C of chroma correction isfixed to C2; if V≧Th2, it is fixed to C1. If Th1<V<Th2, the coefficientC of chroma correction has characteristics in which it linearlydecreases from C2 to C1 with increasing V.

Therefore, the coefficient C of chroma correction is expressed by:

If V≦Th1:C=C2If Th1<V≧Th2:C={(C2−C1)V+(C1×Th1−C2×Th2)}/(Th1−Th2)  (7)

If V≧Th2:C=C1

for Th1<Th2 and C1<C2.

As is known, the area effect of colors occurs up to a visual angle ofabout 20°. Assuming that the visual distance is three times of theheight of a frame, the height of the frame is about 18.9° as visualangle conversion. Therefore, the threshold Th2 preferably assumes avalue obtained by multiplying the number of pixels of an image in thevertical direction by tan 20°/tan 18.9°≈1.06 and raising the product tothe second power. For example, when the number of pixels is 1920(horizontal)×1080 (vertical), Th2 is given by:Th2=(1080×1.06)²≈1,310,000

As the threshold Th1, the threshold Nth used upon determining a regionwith a small number of pixels (S2113) is preferably used.

Also, C1 and C2 can assume arbitrary values as long as C1<C2 holds.However, it is preferable to set C2=1 since the strong chroma correctionis applied to a region with a small apparent size in the chromacorrection processing. In order to skip the chroma correction for aregion with a large apparent size, when the amount of chroma correctionassumes a predetermined value, C1 preferably assumes the reciprocalnumber of the amount of chroma correction.

The coefficient C of chroma correction when Th1<V<Th2 linearly decreasesfrom C2 to C1 with increasing V. Alternatively, C may nonlinearlydecrease as long as it monotonically decreases with increasing V.

Next, the correction coefficient calculator 230 stores the calculatedcoefficient C of chroma correction in an element of the correctioncoefficient storage area corresponding to the pixel appended with alabel number that matches the value (label number) of the counter M(S2305).

The correction coefficient calculator 230 increments the value of thecounter M (S2306), and compares M and Lmax (S2307) If M≦Lmax, theprocess returns to step S2303 to calculate the coefficient C of chromacorrection for another region; if M>Lmax, the processing ends.

Note that the coefficients of chroma correction for respective regionsmay be simultaneously calculated after completion of the calculations ofthe apparent sizes by the size calculator 220. Alternatively, thecoefficient of chroma correction may be calculated every time theapparent size of one region is calculated.

[Filter Processor]

The filter processor 240 applies two-dimensional filtering to thecoefficients C of chroma correction stored in the correction coefficientstorage area. This filtering makes the coefficients C of chromacorrection change smoothly at the boundary of regions since they maychange abruptly at the boundary of regions. Also, this filtering canreduce a large difference between the coefficient C of chroma correctionof a region formed by several pixels (to be referred to as an “isolationregion” hereinafter) and those of surrounding regions. As the type offilter, smoothing filters such as an average value filter, Gaussianfilter, median filter, and the like can be applied. In this embodiment,an average value filter is suitably used in consideration of a reductionof a spatial change in coefficient C of chroma correction and simpleprocessing.

[Chroma Corrector]

FIG. 7 is a flowchart showing the processing of the chroma corrector250. The chroma corrector 250 adjusts the amount of chroma correction tobe applied to each pixel for the purpose of a chroma emphasis effect inaccordance with the coefficient of chroma correction.

The chroma corrector 250 sets the y address to 0 (top end) (S2501) andthe x address to 0 (left end) (S2502).

Next, the chroma corrector 250 acquires an amount E of chroma correctionfrom the memory 201(S2503). The amount E of chroma correction may beeither a uniform value for the entire image or a different value foreach pixel. Also, the amount E of chroma correction may always be apredetermined value or may change according to a user's operation. Whenthe amount E of chroma correction is different for each pixel, theamount E of chroma correction corresponding to the current address (x,y) must be acquired.

The chroma corrector 250 acquires the coefficient C of chroma correctioncorresponding to the address (x, y) from the chroma correction storagearea (S2504), and multiplies the amount E of chroma correction by thecoefficient C of chroma correction (S2505). The corrector 250 applieschroma correction of the product EC to a pixel at the address (x, y)(S2506). When the image data is expressed by lightness, hue, and chromadata, the corrector 250 may multiply the chroma data by the product EC.When the image data is expressed by luminance and color-differencesignals, the corrector 250 may convert color-difference signals Cb andCr so that Cb and Cr become zero for an achromatic color, may multiplythe converted Cb and Cr by the product EC, and then may re-convert themto original color-difference signals.

Equations (8) below show a calculation example for Cb when an image isexpressed by an 8-bit digital signal. The same applies to Cr.Cb2=Cb−128Cb3=Cb2×ECCb′=Cb3+128  (8)where Cb′ is a Cb color-difference signal after correction output fromthe chroma corrector 250.

If the chroma data of a pixel is saturated due to the chroma correction,processing for clipping it to the upper or lower limit value of thechroma data (or color-difference signal) is applied.

The chroma corrector 250 outputs the pixel data that has undergone thechroma correction (S2507), increments the x address (S2508), andcompares the x address value and a maximum value Xmax of the x address(S2509). If x≦Xmax, the process returns to step S2503 to repeat theprocesses in steps S2503 to S2508. If x>Xmax, the corrector 250increments the y address (S2510), and compares the y address value and amaximum value Ymax of the y address (S2511). If y≦Ymax, the processreturns to step S2502 to repeat the processes in steps S2502 to S2510.If y>Ymax, the chroma correction ends.

[Respective Processors and Storage Areas]

FIG. 8 is a block diagram for explaining the relationship between therespective processors and storage areas.

In the region segmentation unit 210, a region segmentation processor 211reads out image data stored in an image storage area 101, and executesregion segmentation. A threshold determination unit 212 compares thenumber N of pixels of a region segmented by the region segmentationprocessor 211 with the threshold Nth. If N≧Nth, the unit 212 outputs thelabel number L of that region to the region segmentation processor 211.If N<Nth, the unit 212 outputs the label number L=0 to the regionsegmentation processor 211. The region segmentation processor 211 storesthe label number L received from the threshold determination unit 212 ina label storage area 102. The region segmentation processor 211 storesregion information of the segmented region in a region informationstorage area 103.

The size calculator 220 acquires region information from the regioninformation storage area 103, calculates the apparent size of the regionbased on the region range data and the number N of pixels, and storesthe calculation result in the region information storage area 103.

The correction coefficient calculator 230 acquires the apparent size Vof the region from the region information storage area 103, andcalculates a coefficient C of chroma correction. The calculator 230searches the label storage area 102 for a pixel appended with the labelnumber of the focused region, and stores the coefficient C of chromacorrection at the corresponding position of a correction coefficientstorage area 104. Therefore, the correction coefficient storage area 104two-dimensionally stores the coefficients C of chroma correction incorrespondence with pixel positions.

The filter processor 240 applies two-dimensional filtering to thecoefficients C of chroma correction stored in the correction coefficientstorage area 104.

The chroma corrector 250 acquires the coefficient C of chroma correctionstored in the correction coefficient storage area 104 for each pixel,and multiplies the amount E of chroma correction by the coefficient C ofchroma correction using a multiplier 251. A chroma correction processor252 reads out image data stored in the image storage area 101 for eachpixel, applies chroma correction according to the output from themultiplier 251, and outputs pixel data that has undergone the chromacorrection.

PROCESSING EXAMPLE 1

FIGS. 9 to 13 show data examples stored in the respective storage areasupon application of the aforementioned image processing to a certainimage. Note that respective parameters are Nth=1, Th1=5, Th2=1310000,C1=0.667, C2=1, and amount E of chroma correction=1.5. Also, in theregion segmentation, the difference ranges of feature amounts withinwhich an adjacent pixel that belongs to the focused region is determinedare Y=±15, Cb=±10, and Cr=±10. Furthermore, filtering of the coefficientC of chroma correction is skipped.

FIG. 9 shows image data of an input image, i.e., YCbCr data stored inthe image storage area 101. Note that FIG. 9 shows 8-bit image datadefined by ITU-RBT.709, data ranges are 16≦Y≦235, 16≦Cb≦240, and16≦Cr≦240, and Cb=Cr=128 represents an achromatic color.

FIG. 10 shows the region segmentation result of the image data shown inFIG. 9, i.e., data stored in the label storage area 102. Numericalvalues described in respective pixels in FIG. 10 are label numbers.

FIG. 11 shows region information of respective regions, i.e., datastored in the region information storage area 103 after the regionsegmentation and calculations of the apparent sizes.

FIG. 11 describes the label number, feature amounts (of an originpixel), addresses of the ends of every directions, number of pixels, andapparent size, but the feature amounts are not indispensable. Forexample, in a region with a label number L=1, the feature amounts of theorigin pixel are Y=120, Cb=201, and Cr=55. This region has an upper leftend (X0, Y0)=(0, 0), a lower right end (X1, Y1)=(270, 380), the number Nof pixels=73201, and an apparent size V=52067.

The number N of pixels of a region with a label number L=2 (region 2) isroughly the same as that of a region with L=107 (region 107). However,these regions have different apparent sizes V, i.e., the apparent sizeV=333,917 for region 2, and V=197,026 for region 107, and region 2 isapparently larger, as can be seen from FIG. 11. This is because thewidth and height of region 2 are W=920 and H=606, while those of region107 are W=461 and H=1192, and region 107 has a flat shape and isapparently small.

FIG. 12 shows coefficients of chroma correction of respective pixels,i.e., data stored in the correction coefficient storage area 104.

FIG. 13 shows image data after chroma correction, i.e., data output fromthe chroma corrector 250. Upon comparison of color-difference signals ofpixel (0, 0) in FIGS. 9 and 13, Cb=201→236 and Cr=55→19, and the chromadata is emphasized. Pixels (0, 0) and (1918, 1078) assume the same valuewhen they are input, but assume different values when they are output.This is because these pixels belong to different regions, and undergochroma correction based on the coefficients C of chroma correction whichare calculated based on the apparent sizes of the respective regions.

Cr of pixel (2, 0) assumes a value less than 16 after chroma correction,but it is clipped to 16 to meet the standard of the image signal.Likewise, since Cb and Cr cannot assume values exceeding 240, if theyassume values exceeding 240 after chroma correction, these values areclipped to 240. When the signal value falls outside the standard range,processing for converting it to fall within a standard range such asgamut mapping and the like may be executed in place of clipping.

In this way, the apparent size is calculated, and the coefficient C ofchroma correction is calculated based on the apparent size to applychroma correction to the image. Therefore, chroma correction suited tothe apparent size can be applied to the focused region. Since eachsegmented region is not limited to that of a specific color such as amemory color or the like, chroma correction suited to the apparent sizecan be applied to a region of an arbitrary color.

PROCESSING EXAMPLE 2

FIGS. 14 to 17 show data examples stored in the respective storage areasupon application of the aforementioned image processing to a certainimage. Note that respective parameters are Nth=8295 (a maximum of 250regions), Th1=8295, Th2=1310000, C1=0.667, C2=l, and amount E of chromacorrection=1.5. Also, in the region segmentation, the difference rangesof feature amounts within which an adjacent pixel that belongs to thefocused region is determined are Y=±15, Cb=±10, and Cr=±10. Furthermore,filtering of the coefficient C of chroma correction is skipped.

Image data of an input image is the same as that shown in FIG. 9.

FIG. 14 shows the region segmentation result of the image data shown inFIG. 9, i.e., data stored in the label storage area 102. Numericalvalues described in respective pixels in FIG. 14 are label numbers.

FIG. 15 shows region information of respective regions, i.e., datastored in the region information storage area 103 after the regionsegmentation and calculations of the apparent sizes.

Upon comparison of Processing Examples 1 and 2, pixels of both a regionwith a label number L=28 (region 28) and a region with L=191 (region191) in Processing Example 1 are appended with a label number L=0 inProcessing Example 2. Since the numbers N of pixels of these regions areless than the threshold Nth=8295, their label numbers L are rewritten to0 in Processing Example 2. Therefore, 202 regions recorded in the regioninformation shown in FIG. 11 are reduced to 22 region in that shown inFIG. 15.

FIG. 16 shows coefficients of chroma correction of respective pixels,i.e., data stored in the correction coefficient storage area 104. SinceProcessing Examples 1 and 2 use different Th1 values, they also havedifferent values of coefficients of chroma correction of respectivepixels. For example, the coefficient of chroma correction of pixel (0,0) is 0.987 in Processing Example 1, but it is 0.989 in ProcessingExample 2.

FIG. 17 shows image data after chroma correction, i.e., data output fromthe chroma corrector 250. Since Processing Examples 1 and 2 usedifferent Th1 values, they also have different values of coefficients ofchroma correction of respective pixels. For example, image data of pixel(1918, 1) is expressed by Y=81, Cb=61, and Cr=160 in Processing Example1, but it is expressed by Y=81, Cb=60, and Cr=160 in Processing Example2.

In this way, small regions and isolation regions each having a smallnumber of pixels are avoided from being labeled (segmented). As aresult, as for an image including many small regions and isolationregions due to noise or the like, a problem of too long calculationprocessing times of the sizes and coefficients of chroma correction dueto too large a number of segmented regions can be solved.

PROCESSING EXAMPLE 3

FIG. 18 shows a data example stored in the correction coefficientstorage area 104 after a certain image undergoes the aforementionedimage processing and filtering. Note that respective parameters areNth=8295 (a maximum of 250 regions), Th1=8295, Th2=1310000, C1=0.667,C2=1, and amount E of chroma correction=1.5. Also, in the regionsegmentation, the difference ranges of feature amounts within which anadjacent pixel that belongs to the focused region is determined areY=±15, Cb=±10, and Cr=±10. Furthermore, filtering of the coefficient Cof chroma correction is executed, and an average value filter is used.

Image data of an input image is the same as that shown in FIG. 9.

As shown in FIG. 14, pixel (2, 1078) belongs to region 0, pixel (1,1078) belongs to region 17, and these pixels define the boundary ofregions. Therefore, in Processing Example 2 (FIG. 16), a coefficient Cof chroma correction=1.000 for pixel (2, 1078), C=0.952 for pixel (1,1078), and their difference Δ is 0.048. By contrast, in ProcessingExample 3 (FIG. 18), C=0.967 for pixel (2, 1078), C=0.963 for pixel (1,1078), and their difference Δ=0.004. That is, in Processing Example 3,the coefficients C of chroma correction are filtered to reduce thedifference Δ between the coefficients C of chroma correction of thepixels which define the boundary to 0.004, thus suppressing an abruptchange in coefficient C of chroma correction at the boundary of theregions.

In this way, by filtering the calculation results of the coefficients ofchroma correction, an abrupt change in coefficient of chroma correctionat the boundary of the regions can be suppressed. Also, the coefficientof chroma correction of a region with a small number of pixels can besuppressed from becoming excessively larger than those of surroundingregions.

According to the first embodiment, chroma correction can be applied inaccordance with the apparent sizes of regions with different featureamounts, i.e., the chroma correction can be applied in consideration ofthe area effect of colors. Also, suited chroma correction can beindividually applied to respective segmented regions. Furthermore,suited chroma correction can be applied without limiting to specificcolors.

Second Embodiment

Image processing according to the second embodiment of the presentinvention will be described hereinafter. Note that the same referencenumerals in the second embodiment denote the same components as those inthe first embodiment, and a detailed description thereof will not begiven.

In the second embodiment, the processing of the size calculator 220 ismodified. That is, the width and height of a central portion of a regionwhose size is to be calculated are calculated using data stored in thelabel storage area 102, and the size of that region is calculated basedon the calculation result.

FIG. 19 is a flowchart showing the processing of the size calculator 220according to the second embodiment.

In the processing shown in FIG. 19, step S2210 of examining the widthand height of a central portion of the focused region is added betweensteps S2203 and S2204 of the processing shown in FIG. 4. In step S2204,the size calculator 220 calculates a size V of the focused region basedon the width Wc and height Hc of the central portion of the region andthe number S of pixels of the region. Note that the second embodimentuses equation (6) to calculate the size V of the region. Alternatively,any other calculation methods may be used as long as they can obtain asize V roughly proportional to the apparent size.

FIGS. 20 to 23 are flowcharts showing the detailed processing of stepS2210.

The size calculator 220 calculates a central position (Xc, Yc) of thefocused region (S2211) by:Xc=int{(X _(R) −X _(L))/2}+1Yc=int{(Y _(B) −Y _(T))/2}+1  (9)where int is a function of truncating fractions,

X_(R) is the x address of the right end of the region,

X_(L) is the x address of the left end of the region,

Y_(B) is the y address of the bottom end of the region, and

Y_(T) is the y address of the top end of the region.

The size calculator 220 examines pixels from the central position (Xc,Yc) of the region in the upper, lower, right, and left directions tocalculate the width Wc and height Hc of the region central portion. Forthis purpose, the calculator 220 sets Yc in a variable Py (S2212), andXc in a variable Px (S2213). The calculator 220 then checks if Px>0(S2214). As a result of checking, if Px=0, since that position is theleft end position of the central portion of the focused region, theprocess jumps to step S2218.

If Px>0, the size calculator 220 decrements Px (S2215), and checks ifthe label number which is stored in the label storage area 102 and isappended to pixel (Px, Py) matches the label number L of the focusedregion (S2216). As a result of checking, if the two label numbers match,the process returns to step S2214. If the two label numbers aredifferent, the size calculator 220 increments Px (S2217), and theprocess advances to step S2218.

The size calculator 220 sets Px in a left end position X_(CL) of thecentral portion of the focused region (S2218).

The size calculator 220 sets Px=Xc again (S2219) and checks if Px<Xmax(the maximum value of the x address) (S2220). As a result of checking,if Px=Xmax, since that position is the right end position of the centralportion of the focused region, the process jumps to step S2224.

On the other hand, if Px<Xmax, the size calculator 220 increments Px(S2221), and checks if the label number which is stored in the labelstorage area 102 and is appended to pixel (Px, Py) matches the labelnumber L of the focused region (S2222). As a result of checking, if thetwo label numbers match, the process returns to step S2220. If the twolabel numbers are different, the size calculator 220 decrements Px(S2223), and the process advances to step S2224.

Next, the size calculator 220 sets Px in a right end position X_(CR) ofthe central portion of the focused region (S2224).

The size calculator 220 sets Px=Xc again (S2225) and checks if Py>0(S2226). As a result of checking, if Py=0, since that position is thetop end position of the central portion of the focused region, theprocess jumps to step S2230.

If Py>0, the size calculator 220 decrements Py (S2227), and checks ifthe label number which is stored in the label storage area 102 and isappended to pixel (Px, Py) matches the label number L of the focusedregion (S2228). As a result of checking, if the two label numbers match,the process returns to step S2226. On the other hand, if the two labelnumbers are different, the size calculator 220 increments Py (S2229),and the process advances to step S2230.

The size calculator 220 sets Py in a top end position Y_(CT) of thecentral portion of the focused region (S2230).

The size calculator 220 sets Py=Yc again (S2231), and checks if Py<Ymax(the maximum value of the y address (S2232). As a result of checking, ifPy=Ymax, since that position is the bottom end position of the centralportion of the focused region, the process jumps to step S2236.

On the other hand, if Py<Ymax, the size calculator 220 increments Py(S2233), and checks the label number which is stored in the labelstorage area 102 and is appended to pixel (Px, Py) matches the labelnumber L of the focused region (S2234). As a result of checking, if thetwo label numbers match, the process returns to step S2232. If the twolabel numbers are different, the size calculator 220 decrements Py(S2235) and the process advances to step S2236.

The size calculator 220 sets Py in a bottom end position Y_(CB) of thecentral portion of the focused region (S2236).

The size calculator 220 calculates the width Wc and height Hc of theregion central portion (S2237) by:WC=X _(CR) −X _(CL)+1Hc=Y _(CB) −Y _(CT)+1  (10)

Third Embodiment

Image processing according to the third embodiment of the presentinvention will be described hereinafter. Note that the same referencenumerals in the third embodiment denote the same components as those inthe first and second embodiments, and a detailed description thereofwill not be given.

In the third embodiment, the processing of the region segmentation unit210 is modified. That is, in the first embodiment, the number N ofpixels of the focused region and region range data are acquired whilesegmenting one region. However, in the third embodiment, the number N ofpixels and region range data are acquired after completion ofsegmentation of one region.

FIG. 24 is a flowchart showing the processing of the region segmentationunit 210 according to the third embodiment.

Referring to FIG. 24, the region segmentation unit 210 segments oneregion in step S2121. As the region segmentation method, for example, aregion growing method, statistical hypothesis testing method,integration method, separation method, separation/integration method,K-means method, and every other region segmentation methods can be used.Especially, since counting of the number of pixels of each region andacquisition of the region range data are done after region segmentation,the region segmentation method is not particularly limited.

After one region is segmented, the region segmentation unit 210 acquiresregion range data in step S2122. The unit 210 acquires the region rangedata by raster scanning from the top end and left end of an image, butmay be use any other methods.

The region segmentation unit 210 acquires the number N of pixels of theregion in step S2123. The unit 210 acquires the number N of pixels byraster-scanning from the top end and left end of an image, but may beuse any other methods.

Since other processes are the same as those in the first embodiment, adetailed description thereof will not be given.

Exemplary Embodiments

The present invention can be applied to a system constituted by aplurality of devices (e.g., host computer, interface, reader, printer)or to an apparatus comprising a single device (e.g., copying machine,facsimile machine).

Further, the present invention can provide a storage medium storingprogram code for performing the above-described processes to a computersystem or apparatus (e.g., a personal computer), reading the programcode, by a CPU or MPU of the computer system or apparatus, from thestorage medium, then executing the program.

In this case, the program code read from the storage medium realizes thefunctions according to the embodiments.

Further, the storage medium, such as a floppy disk, a hard disk, anoptical disk, a magneto-optical disk, CD-ROM, CD-R, a magnetic tape, anon-volatile type memory card, and ROM can be used for providing theprogram code.

Furthermore, besides above-described functions according to the aboveembodiments can be realized by executing the program code that is readby a computer, the present invention includes a case where an OS(operating system) or the like working on the computer performs a partor entire processes in accordance with designations of the program codeand realizes functions according to the above embodiments.

Furthermore, the present invention also includes a case where, after theprogram code read from the storage medium is written in a functionexpansion card which is inserted into the computer or in a memoryprovided in a function expansion unit which is connected to thecomputer, CPU or the like contained in the function expansion card orunit performs a part or entire process in accordance with designationsof the program code and realizes functions of the above embodiments.

In a case where the present invention is applied to the aforesaidstorage medium, the storage medium stores program code corresponding tothe flowcharts described in the embodiments.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modification and equivalent structures and functions.

This application claims the benefit of Japanese Patent Application No.2006-242145, filed Sep. 6, 2006, which is hereby incorporated byreference herein in its entirety.

1. An image processing apparatus comprising: a segmentation unit,arranged to segment an input image into a plurality of regions as setsof pixels having similar feature amounts; a calculator, arranged tocalculate a size evaluation value of each region segmented by saidsegmentation unit based on the number of pixels and a shape of the eachsegmented region; and a corrector, arranged to apply chroma correctionto the input image to weaken correction as a region has a larger sizeevaluation value.
 2. The apparatus according to claim 1, wherein saidcalculator uses dimensions in a vertical direction and a horizontaldirection of the segmented region as the shape.
 3. The apparatusaccording to claim 1, wherein said calculator uses dimensions in avertical direction and a horizontal direction of a central portion ofthe segmented region as the shape.
 4. The apparatus according to claim1, wherein said segmentation unit executes the segmentation usinginformation associated with colors of the image as the feature amounts.5. The apparatus according to claim 1, wherein said correctorindividually applies the chroma correction to the regions segmented bysaid segmentation unit.
 6. The apparatus according to claim 1, whereinwhen the number of pixels of the each segmented region is smaller thanthe predetermined number of pixels, said segmentation unit classifiespixels of that region which do not belong to any region, and saidcalculator sets the size evaluation values of a set of pixels which donot belong to any region to be zero.
 7. The apparatus according to claim1, wherein said corrector comprising: a coefficient calculator, arrangedto calculate coefficients of chroma correction of the region forrespective pixels in accordance with the size evaluation value of eachregion, and to store the coefficients of chroma correction in a memory;and a filter, arranged to apply two-dimensional filtering to thecoefficients of chroma correction stored in the memory.
 8. An imageprocessing apparatus comprising: a specifying unit, arranged to specifya region as a set of pixels, each of which has the similar featureamount, from an input image; a calculator, arranged to calculate a sizeevaluation value of the specified region based on the number of pixelsand a shape of the specified region; and a corrector, arranged to applychroma correction to the input image to weaken correction as a regionhas a larger size evaluation value, wherein the size evaluation valuegrows as large as the number of pixels of the specified regionincreases, and the size evaluation value grows as large as the shape ofthe specified region approaches a round shape.
 9. An image processingmethod comprising the steps of: segmenting an input image into aplurality of regions as sets of pixels having similar feature amounts;calculating a size evaluation value of each segmented region based onthe number of pixels and a shape of the each segmented region; andapplying chroma correction to the input image to weaken correction as aregion has a larger size evaluation value.
 10. The method according toclaim 9, wherein the calculating step uses dimensions in a verticaldirection and a horizontal direction of the segmented region as theshape.
 11. The method according to claim 9, wherein the calculating stepuses dimensions in a vertical direction and a horizontal direction of acentral portion of the segmented region as the shape.
 12. The methodaccording to claim 9, wherein the segmenting step executes thesegmentation using information associated with colors of the image asthe feature amounts.
 13. The method according to claim 9, wherein thecorrecting step individually applies the chroma correction to theregions segmented in the segmenting step.
 14. The method according toclaim 9, wherein when the number of pixels of the each segmented regionis smaller than the predetermined number of pixels, the segmenting stepclassifies pixels of that region which do not belong to any region, andthe calculating step sets the size evaluation values of a set of pixelswhich do not belong to any region to be zero.
 15. The method accordingto claim 9, wherein the correcting step comprising the steps of:calculating coefficients of chroma correction of the region forrespective pixels in accordance with the size evaluation value of eachregion, and storing the coefficients of chroma correction in a memory;and applying two-dimensional filtering to the coefficients of chromacorrection stored in the memory.
 16. An image processing methodcomprising the steps of: specifying a region as a set of pixels, each ofwhich has the similar feature amount, from an input image; calculating asize evaluation value of the specified region based on the number ofpixels and a shape of the specified region; and applying chromacorrection to the input image to weaken correction as a region has alarger size evaluation value, wherein the size evaluation value grows aslarge as the number of pixels of the specified region increases, and thesize evaluation value grows as large as the shape of the specifiedregion approaches a round shape.